The present invention relates to steganography and more particularly to inserting digital watermarks in images and to detecting and reading such watermarks.
The technology for inserting digital watermarks into images is well developed. For example see issued U.S. Pat. No. 5,862,260 (Rhoads), U.S. Pat. No. 5,930,369 (Cox), U.S. Pat. No. 5,905,800 (Moskowitz), and U.S. Pat. No. 6,122,403 (Rhoads) and co-pending application Ser. No. 09/553,084 filed Apr. 19, 2000 (now U.S. Pat. No. 6,590,996), all of which are hereby incorporated herein by reference. Watermarking technology is also included in some commercially available image editing programs such as xe2x80x9cAdobe Photoshopxe2x80x9d which is marketed by Adobe Corporation of San Jose Calif. and xe2x80x9cCorel Drawxe2x80x9d which is marked by Corel Corporation of Ontario Canada.
Colored Images are generally stored in computers using a RGB (Red, Green, Blue) format. The above referenced commercially available programs insert a watermark into an RGB image by modifying the luminance value of pixels in the image. The value of each color plane in the image is modified by the known relationship between overall luminance and the value of each color plane. For example, it is known that the ratio between overall luminance and the value of the colors in an RGB image is 0.3 for Red, 0.6 for Green and 0.1 for Blue.
In many situations better results can be achieved if the watermark is inserted adaptively. That is, if the intensity of the watermark inserted in each particular area of the image is adjusted in accordance with the data hiding attributes associated with that particular area of the imager. The above referenced commercial image editing programs insert watermarks into images adaptively. U.S. Pat. No. 6,590,996 filed Apr. 19, 2000 (which is incorporated herein by reference) describes adapting the watermarking process to the color of an image.
Colored images are printed using multiple printing plates. The image is divided into color planes corresponding to the colors of ink used to print the image. Each color is printed using a separate plate which prints that color. For example an image may be separated into Cyan, Magenta, Yellow and Black (CMYK) color planes. A separate plate is used to print each color. The different plates must be precisely aligned. Any misalignment of the plates will cause blurring in the image and may make it difficult or impossible to read a watermark that was embedded in the image.
That is, when an image contains digital watermark data in each color plane, misalignment of the plates used to print different colors can cause the watermark data in one color plane to, in effect, cancel the watermark data in a different color plane.
FIG. 1 illustrates the main steps in an image watermarking process. The process begins with an image 101 which has RGB (red, green blue) values for each pixel in the image. The object of the process is to insert watermark payload data 102 into image 101. The change (or tweak) for the luminance of each pixel in image 101 which will insert the payload 102 into the image is calculated. An example of how tweak values can be calculated is shown in the above referenced issued U.S. patents and in other publicly available literature. The tweak values for the luminance are changed into changes in RGB values as indicated by block 105. The transformation is done according to the known relationship between color values and luminance. Generally the luminance of a pixel can be approximated as 0.3 times the red value plus 0.6 times the green value plus 0.1 times the blue value.
The color values of each pixel in the image 101 are changed by the calculated amounts as shown in FIG. 1B in order to watermark the image. Watermark data tile 116 specifies the amount of change in luminosity for each pixel in a square array of pixels. The pixels in the image 101 are divided into an array of squares that have the same size as the tile 116. The amount that the pixels in image 101 are changed is adapted to the characteristics of the image. For example consider two squares 118a and 118b in the image 101. If the characteristics in square 118a are such that it can carry less watermark signal than the characteristics of the image in square 118b, the pixels in square 118a may only be changed by one half of the amount specified in tile 116 and the pixels in square 118b may be changed by the full amount specified in the tile 116. The technology for adaptively inserting watermark signals in an image using various techniques is known.
Finally, as indicated by block 107, the RGB colors are changed into CMYK (cyan, magenta, yellow, and black) values for printing, and each color is printed with a separate plate as indicated by block 108. If the plates used to print the different colors are misaligned, the watermark in one color can effectively cancel the watermark in another color.
Printing with misaligned plates is illustrated in FIG. 2, which shows (greatly exaggerated and simplified) the areas printed by different plates for two pixels in the image. To facilitate illustration and explanation, in FIG. 2, rectangles are used to designate the area printed by a first color plate and the circles are used to designate areas printed by a different color plate. The areas are the size of one pixel in the image. It is noted that in a typical printing process the areas would have the same shape. They are shown here as having different shapes for ease of illustration and explanation. It is also noted that each pixel area would normally contain multiple ink dots. The actual ink dots are not shown in FIG. 2. As illustrated in FIG. 2, square 201C represents the cyan printing from one pixel in the image and circle 201Y represents the yellow printing from this same pixel. The same applies to square 202C and circle 202Y. If the plates where aligned, the circles and squares would be directly on top of each other. However, in the illustration shown in FIG. 2, due to misalignment of the printing plates, the circles and squares are not aligned. Let us assume that the watermarking process increased the luminance of the pixel from which square 201C and circle 201Y originated and decreased the luminance of the adjacent pixel. Since the plates were misaligned, in the area where circle 201Y overlaps square 202C, the plus increment in circle 201Y would cancel the negative decrement in square 202C. Thus, misalignment of printing plates can have the effect of at least partially canceling watermark data in an image. In most situations the misalignment would not completely cancel the watermark data; however, it would weakens the watermark signal.
One aspect of the present invention is directed to minimizing the effect of plate misalignment on the detectability and readability of watermark data in an image. The effect of misalignment of the printing plates is minimized by detecting the dominant color in an image and inserting a watermark only into that color plane of the image. In another embodiment of the invention, the image is divided into regions and the dominant color in each region is determined. In each region the watermark data is inserted into the dominant color in that region. In still another embodiment of the invention, a first watermark is inserted into the dominant color plane of an image and a second watermark is inserted in one of the other color planes of the image. In still another embodiment of the invention the dominant color of the entire image (or of a region of the image) is detected and the watermark is inserted into the dominant color if that color is on the yellow-blue axis. If the dominant color is not on the yellow-blue axis, the watermark is inserted into the strongest of the secondary colors.
In a situation where an image is watermarked by first separating the image into color planes and inserting a watermark into one or more color plane, it is easier to detect and read the watermark if the image is first separated into the same color planes as the color planes used during the embedding and printing operation. If one knows the color planes into which the image was divided during the insertion process, during the detection and reading process one would divide the image into these same colors before performing the detection and reading operation.
However, if one does not know the color planes into which the image was divided before the watermark was inserted (that is, if one does not know the color of the inks used to print the image), one can approximately determine the colors of the inks used to print the image by: First translating the image into a representation such as for example a HSI (hue saturation intensity) and next making a histogram of the hue values of the pixels in the image. The dominant hues represent an approximation of the colors used to print the image. The image can then be divided into these colors and the watermarks read from these colors. The histogram can also be made using other similar representations such as HSV, YUV, YcrCb, etc.
Multiple watermarks can be embedded into a single image by first dividing the image into multiple color planes, and by embedding a different watermark in each color plane. If the orientation of the watermark introduced into each color plane differs from the orientation of the watermarks in the other color planes, any noise created by miss-registration of the printing plates is non-correlated and thus the watermark in one color would not interfere with reading the watermark from a different color. Thus, when introducing watermarks in more than one color plane, the watermarks in the different color planes can be introduced in different orientations to minimize noise.
It is also noted that when an image is rasterized for printing, the rasterization for each color plane is typically done in a different direction. For example if the CMYK colors are used, the orientations might be Cyan 0 degrees, Magenta 45 degrees, Yellow 15 degrees, and black 75 degrees. The orientations are set by the printer. Better detection can be achieved if the watermark is introduced in each color plane at the same angle as the angle used to print that color plane. In this way, the rasterization proceeds parallel to the watermark orientation.